{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f2f70ec6",
   "metadata": {},
   "source": [
    "### [模型组网](https://www.paddlepaddle.org.cn/documentation/docs/zh/guides/02_paddle2.0_develop/04_model_cn.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "740a812a",
   "metadata": {},
   "source": [
    "paddle中的组网方式有两种：sequential 和 subclass 模式"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b1e3a698",
   "metadata": {},
   "outputs": [],
   "source": [
    "import paddle\n",
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "876d7dc9",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "822fc4c6",
   "metadata": {},
   "source": [
    "#### sequential 组网\n",
    ">针对顺序的线性网络结构，使用sequential可以快速组网"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "f8e7c9b0",
   "metadata": {},
   "outputs": [],
   "source": [
    "mnist = paddle.nn.Sequential(\n",
    "    paddle.nn.Flatten(),\n",
    "    paddle.nn.Linear(784,512),\n",
    "    paddle.nn.ReLU(),\n",
    "    paddle.nn.Dropout(0.2),\n",
    "    paddle.nn.Linear(512,10)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5b632e4",
   "metadata": {},
   "source": [
    "#### subclass 组网\n",
    ">复杂的网络结构，使用Layer子类来定义模型\n",
    ">\n",
    ">在 init 中编写需要的层\n",
    ">\n",
    ">在 forward 中声明层的前向计算方式"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "2b4ada69",
   "metadata": {},
   "outputs": [],
   "source": [
    "class Mnist(paddle.nn.Layer):\n",
    "    def __init__(self):\n",
    "        super(Mnist, self).__init__()\n",
    "        self.flatten = paddle.nn.Flatten()\n",
    "        self.linear_1 = paddle.nn.Linear(784, 512)\n",
    "        self.linear_2 = paddle.nn.Linear(512, 10)\n",
    "        self.relu = paddle.nn.ReLU()\n",
    "        self.dropout = paddle.nn.Dropout(0.2)\n",
    "        \n",
    "    def forward(self,inputs):\n",
    "        y = self.flatten(inputs)\n",
    "        y = self.linear_1(y)\n",
    "        y = self.relu(y)\n",
    "        y = self.dropout(y)\n",
    "        y = self.linear_2(y)\n",
    "        return y\n",
    "\n",
    "mnist = Mnist()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "06745df8",
   "metadata": {},
   "source": [
    "#### 飞浆内置的模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "cb849a37",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "飞浆内置的模型： ['ResNet', 'resnet18', 'resnet34', 'resnet50', 'resnet101', 'resnet152', 'VGG', 'vgg11', 'vgg13', 'vgg16', 'vgg19', 'MobileNetV1', 'mobilenet_v1', 'MobileNetV2', 'mobilenet_v2', 'LeNet']\n"
     ]
    }
   ],
   "source": [
    "print(\"飞浆内置的模型：\",paddle.vision.models.__all__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "9f2880da",
   "metadata": {},
   "outputs": [],
   "source": [
    "lenet = paddle.vision.models.LeNet()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1be044b",
   "metadata": {},
   "source": [
    "使用 summary 查看模型的结构"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "48ee701f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------------------------------------------------------------------\n",
      " Layer (type)       Input Shape          Output Shape         Param #    \n",
      "===========================================================================\n",
      "   Conv2D-1      [[64, 1, 28, 28]]     [64, 6, 28, 28]          60       \n",
      "    ReLU-3       [[64, 6, 28, 28]]     [64, 6, 28, 28]           0       \n",
      "  MaxPool2D-1    [[64, 6, 28, 28]]     [64, 6, 14, 14]           0       \n",
      "   Conv2D-2      [[64, 6, 14, 14]]     [64, 16, 10, 10]        2,416     \n",
      "    ReLU-4       [[64, 16, 10, 10]]    [64, 16, 10, 10]          0       \n",
      "  MaxPool2D-2    [[64, 16, 10, 10]]     [64, 16, 5, 5]           0       \n",
      "   Linear-5         [[64, 400]]           [64, 120]           48,120     \n",
      "   Linear-6         [[64, 120]]            [64, 84]           10,164     \n",
      "   Linear-7          [[64, 84]]            [64, 10]             850      \n",
      "===========================================================================\n",
      "Total params: 61,610\n",
      "Trainable params: 61,610\n",
      "Non-trainable params: 0\n",
      "---------------------------------------------------------------------------\n",
      "Input size (MB): 0.19\n",
      "Forward/backward pass size (MB): 7.03\n",
      "Params size (MB): 0.24\n",
      "Estimated Total Size (MB): 7.46\n",
      "---------------------------------------------------------------------------\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'total_params': 61610, 'trainable_params': 61610}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "paddle.summary(lenet,(64,1,28,28))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4db932a1",
   "metadata": {},
   "source": [
    "### 训练与预测验证\n",
    ">第一种：使用 paddle.Model 对模型进行封装，使用高层API进行训练和预测，比如：model.fit()、model.predict()、model.evaluate()、model.summary()等\n",
    ">\n",
    ">第二种：直接用基础API进行训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "6f7986f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "import paddle\n",
    "from paddle.vision.transforms import ToTensor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "63f6aef6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 加载数据\n",
    "train_dataset = paddle.vision.datasets.MNIST(mode=\"train\",transform=ToTensor())\n",
    "test_dataset = paddle.vision.datasets.MNIST(mode=\"test\",transform=ToTensor())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "87ada790",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(60000, 10000)"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(train_dataset),len(test_dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0958992b",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "2c5b05eb",
   "metadata": {},
   "source": [
    "#### 使用 paddle.model 训练预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "6239770e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 使用简单的 sequential 组网\n",
    "mnist = paddle.nn.Sequential(\n",
    "    paddle.nn.Flatten(1,-1),\n",
    "    paddle.nn.Linear(784,512),\n",
    "    paddle.nn.ReLU(),\n",
    "    paddle.nn.Dropout(0.2),\n",
    "    paddle.nn.Linear(512,10)\n",
    ")\n",
    "\n",
    "# 使用 paddle.model 封装模型\n",
    "model = paddle.Model(mnist)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "063a9187",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 使用高层 API 进行训练前的准备\n",
    "model.prepare(\n",
    "    optimizer=paddle.optimizer.Adam(parameters=model.parameters()),\n",
    "    loss=paddle.nn.CrossEntropyLoss(),\n",
    "    metrics=paddle.metric.Accuracy()\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "caa503b1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The loss value printed in the log is the current step, and the metric is the average value of previous steps.\n",
      "Epoch 1/5\n",
      "step 938/938 [==============================] - loss: 0.0969 - acc: 0.9284 - 29ms/step          \n",
      "Epoch 2/5\n",
      "step 938/938 [==============================] - loss: 0.0410 - acc: 0.9688 - 27ms/step          \n",
      "Epoch 3/5\n",
      "step 938/938 [==============================] - loss: 0.0214 - acc: 0.9780 - 26ms/step          \n",
      "Epoch 4/5\n",
      "step 938/938 [==============================] - loss: 0.0052 - acc: 0.9819 - 27ms/step          \n",
      "Epoch 5/5\n",
      "step 938/938 [==============================] - loss: 0.0929 - acc: 0.9861 - 27ms/step          \n"
     ]
    }
   ],
   "source": [
    "model.fit(\n",
    "    train_dataset,\n",
    "    epochs=5,\n",
    "    batch_size=64,\n",
    "    verbose=1\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "4035a82c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Eval begin...\n",
      "step 10000/10000 [==============================] - loss: 1.7881e-06 - acc: 0.9796 - 3ms/step          - loss\n",
      "Eval samples: 10000\n"
     ]
    }
   ],
   "source": [
    "eval_result = model.evaluate(test_dataset,verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "5d9fc54e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predict begin...\n",
      "step 10000/10000 [==============================] - 4ms/step          \n",
      "Predict samples: 10000\n"
     ]
    }
   ],
   "source": [
    "test_result = model.predict(test_dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "95c6e53d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# test_result"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a22d78bb",
   "metadata": {},
   "source": [
    "#### 通过基础API实现模型的训练与预测"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "fe86a595",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 采用 subclass 组网\n",
    "class Mnist(paddle.nn.Layer):\n",
    "    def __init__(self):\n",
    "        super(Mnist,self).__init__()\n",
    "        self.flatten = paddle.nn.Flatten()\n",
    "        self.linear_1 = paddle.nn.Linear(784,512)\n",
    "        self.linear_2 = paddle.nn.Linear(512,10)\n",
    "        self.relu = paddle.nn.ReLU()\n",
    "        self.dropout = paddle.nn.Dropout(0.2)\n",
    "        \n",
    "    def forward(self,inputs):\n",
    "        y = self.flatten(inputs)\n",
    "        y = self.linear_1(y)\n",
    "        y = self.relu(y)\n",
    "        y = self.dropout(y)\n",
    "        y = self.linear_2(y)\n",
    "        return y\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "2d78d0ef",
   "metadata": {},
   "outputs": [],
   "source": [
    "# dataset 数据集的定义不变，用 DataLoader 加载数据\n",
    "train_loader = paddle.io.DataLoader(train_dataset,batch_size=64,shuffle=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b242ec3",
   "metadata": {},
   "source": [
    ">train_loader 总共有 938 个数据集，每个数据集中有 64 个样本\n",
    ">\n",
    ">采用异步加载数据，所以每调用一次返回一个数据集\n",
    ">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "f846f2d7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "938"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(train_loader)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "2255d92d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch：0,batch_id：900,loss is：[0.145264]，acc is：[0.953125]\n",
      "epoch：1,batch_id：900,loss is：[0.05669905]，acc is：[0.984375]\n",
      "epoch：2,batch_id：900,loss is：[0.05993386]，acc is：[0.984375]\n",
      "epoch：3,batch_id：900,loss is：[0.07510052]，acc is：[0.953125]\n",
      "epoch：4,batch_id：900,loss is：[0.02449301]，acc is：[0.984375]\n"
     ]
    }
   ],
   "source": [
    "mnist = Mnist()\n",
    "# 开启训练模式\n",
    "mnist.train()\n",
    "# 设置迭代次数\n",
    "epochs = 5\n",
    "# 设置优化器\n",
    "optim = paddle.optimizer.Adam(parameters=mnist.parameters())\n",
    "# 设置损失函数\n",
    "loss_fn = paddle.nn.CrossEntropyLoss()\n",
    "\n",
    "for epoch in range(epochs):\n",
    "    # 通过加载器每次随即返回64条数据\n",
    "    for batch_id,data in enumerate(train_loader()):\n",
    "        x_data = data[0]\n",
    "        y_data = data[1]\n",
    "        predicts = mnist(x_data)\n",
    "        # 计算损失,等价于 prepare 中的 loss 设置\n",
    "        loss = loss_fn(predicts,y_data)\n",
    "        # 计算准确率，等价于 prepare 中的 metrics 设置\n",
    "        acc = paddle.metric.accuracy(predicts,y_data)\n",
    "        # 下面的 反向传播、更新参数、梯度清零都被封装到 Model.fit() 中\n",
    "        \n",
    "        # 反向传播\n",
    "        loss.backward()\n",
    "        if (batch_id+1)%900 ==0:\n",
    "            print(\"epoch：{},batch_id：{},loss is：{}，acc is：{}\".format(epoch,batch_id+1,loss.numpy(),acc.numpy()))\n",
    "        \n",
    "        # 更新参数\n",
    "        optim.step()\n",
    "        # 梯度清零\n",
    "        optim.clear_grad()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76d2a033",
   "metadata": {},
   "source": [
    "#### 用基础API验证模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "51f2e827",
   "metadata": {},
   "outputs": [],
   "source": [
    "test_loader = paddle.io.DataLoader(test_dataset,batch_size=60,drop_last=True)\n",
    "loss_fn = paddle.nn.CrossEntropyLoss()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "8cf5ea75",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "batch_id：30，loss is [0.05411525]，acc is：[0.98333335]\n",
      "batch_id：60，loss is [0.11626275]，acc is：[0.98333335]\n",
      "batch_id：90，loss is [0.00137621]，acc is：[1.]\n",
      "batch_id：120，loss is [0.00256267]，acc is：[1.]\n",
      "batch_id：150，loss is [0.01000697]，acc is：[1.]\n"
     ]
    }
   ],
   "source": [
    "# 开启验证\n",
    "mnist.eval()\n",
    "for batch_id,data in enumerate(test_loader()):\n",
    "    x_data = data[0]\n",
    "    y_data = data[1]\n",
    "    predicts = mnist(x_data)\n",
    "    loss = loss_fn(predicts,y_data)\n",
    "    acc = paddle.metric.accuracy(predicts,y_data)\n",
    "    # 打印消息\n",
    "    if (batch_id+1)%30==0:\n",
    "        print(\"batch_id：{}，loss is {}，acc is：{}\".format(batch_id+1,loss.numpy(),acc.numpy()))\n",
    "    \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f665e36d",
   "metadata": {},
   "source": [
    "#### 用基础API预测模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "b231e831",
   "metadata": {},
   "outputs": [],
   "source": [
    "test_loader = paddle.io.DataLoader(test_dataset,batch_size=64,drop_last=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "4e12cd4d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "predict finished\n"
     ]
    }
   ],
   "source": [
    "mnist.eval()\n",
    "for batch_id,data in enumerate(test_loader()):\n",
    "    x_data = data[0]\n",
    "    predicts = mnist(x_data)\n",
    "print(\"predict finished\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2f0082d9",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
